Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Minari datasets #1721

Merged
merged 20 commits into from
Dec 4, 2023
Merged

[Feature] Minari datasets #1721

merged 20 commits into from
Dec 4, 2023

Conversation

vmoens
Copy link
Contributor

@vmoens vmoens commented Nov 29, 2023 •
edited
Loading

Inegrates Minari in our dataset hub

Usage:

from torchrl.data.datasets.minari import MinariExperienceReplay

data = MinariExperienceReplay("door-human-v1", batch_size=32)
for sample in data:
    print(sample)
    break

which prints

TensorDict(
    fields={
        action: Tensor(shape=torch.Size([32, 28]), device=cpu, dtype=torch.float32, is_shared=False),
        index: Tensor(shape=torch.Size([32]), device=cpu, dtype=torch.int64, is_shared=False),
        info: TensorDict(
            fields={
                success: Tensor(shape=torch.Size([32]), device=cpu, dtype=torch.bool, is_shared=False)},
            batch_size=torch.Size([32]),
            device=None,
            is_shared=False),
        next: TensorDict(
            fields={
                observation: Tensor(shape=torch.Size([32, 39]), device=cpu, dtype=torch.float64, is_shared=False),
                reward: Tensor(shape=torch.Size([32, 1]), device=cpu, dtype=torch.float64, is_shared=False),
                state: TensorDict(
                    fields={
                        door_body_pos: Tensor(shape=torch.Size([32, 3]), device=cpu, dtype=torch.float64, is_shared=False),
                        qpos: Tensor(shape=torch.Size([32, 30]), device=cpu, dtype=torch.float64, is_shared=False),
                        qvel: Tensor(shape=torch.Size([32, 30]), device=cpu, dtype=torch.float64, is_shared=False)},
                    batch_size=torch.Size([32]),
                    device=None,
                    is_shared=False),
                terminated: Tensor(shape=torch.Size([32, 1]), device=cpu, dtype=torch.bool, is_shared=False),
                truncated: Tensor(shape=torch.Size([32, 1]), device=cpu, dtype=torch.bool, is_shared=False)},
            batch_size=torch.Size([32]),
            device=None,
            is_shared=False),
        observation: Tensor(shape=torch.Size([32, 39]), device=cpu, dtype=torch.float64, is_shared=False),
        state: TensorDict(
            fields={
                door_body_pos: Tensor(shape=torch.Size([32, 3]), device=cpu, dtype=torch.float64, is_shared=False),
                qpos: Tensor(shape=torch.Size([32, 30]), device=cpu, dtype=torch.float64, is_shared=False),
                qvel: Tensor(shape=torch.Size([32, 30]), device=cpu, dtype=torch.float64, is_shared=False)},
            batch_size=torch.Size([32]),
            device=None,
            is_shared=False)},
    batch_size=torch.Size([32]),
    device=None,
    is_shared=False)

TODO:

  • Tests across several datasets
  • Decide what to do with text data
  • integrate metadata in the buffer class (and figure out how to save them without recurring to pickle)
    The plan would be to save things as a json after converting the specs to torchrl format

cc @rodrigodelazcano @WillDudley @younik @BY571

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Nov 29, 2023 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1721

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (17 Unrelated Failures)

As of commit 0090fe0 with merge base d545364 (image):

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 29, 2023
@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 29, 2023 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 64.0120ms 62.7584ms 15.9341 Ops/s 15.0018 Ops/s $\textbf{\color{#35bf28}+6.21\%}$
test_sync 40.3020ms 34.0208ms 29.3938 Ops/s 26.9923 Ops/s $\textbf{\color{#35bf28}+8.90\%}$
test_async 58.1312ms 32.6184ms 30.6576 Ops/s 30.5066 Ops/s $\color{#35bf28}+0.49\%$
test_simple 0.4861s 0.4319s 2.3156 Ops/s 2.2391 Ops/s $\color{#35bf28}+3.41\%$
test_transformed 0.6426s 0.5934s 1.6853 Ops/s 1.6404 Ops/s $\color{#35bf28}+2.74\%$
test_serial 1.3772s 1.3355s 0.7488 Ops/s 0.7355 Ops/s $\color{#35bf28}+1.81\%$
test_parallel 1.3705s 1.2991s 0.7698 Ops/s 0.7843 Ops/s $\color{#d91a1a}-1.85\%$
test_step_mdp_speed[True-True-True-True-True] 0.1881ms 22.7169μs 44.0202 KOps/s 44.1618 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-True-True-True-False] 80.6730μs 13.4385μs 74.4133 KOps/s 73.5029 KOps/s $\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-True-True-False-True] 41.6380μs 13.8629μs 72.1348 KOps/s 71.4287 KOps/s $\color{#35bf28}+0.99\%$
test_step_mdp_speed[True-True-True-False-False] 56.7350μs 8.1495μs 122.7076 KOps/s 120.6534 KOps/s $\color{#35bf28}+1.70\%$
test_step_mdp_speed[True-True-False-True-True] 59.7320μs 23.9453μs 41.7619 KOps/s 41.1156 KOps/s $\color{#35bf28}+1.57\%$
test_step_mdp_speed[True-True-False-True-False] 51.1560μs 14.8894μs 67.1618 KOps/s 66.8581 KOps/s $\color{#35bf28}+0.45\%$
test_step_mdp_speed[True-True-False-False-True] 47.3990μs 15.1311μs 66.0892 KOps/s 64.2442 KOps/s $\color{#35bf28}+2.87\%$
test_step_mdp_speed[True-True-False-False-False] 35.4770μs 9.4989μs 105.2758 KOps/s 103.3998 KOps/s $\color{#35bf28}+1.81\%$
test_step_mdp_speed[True-False-True-True-True] 89.0870μs 25.1894μs 39.6993 KOps/s 38.6878 KOps/s $\color{#35bf28}+2.61\%$
test_step_mdp_speed[True-False-True-True-False] 51.0960μs 16.0424μs 62.3349 KOps/s 60.6123 KOps/s $\color{#35bf28}+2.84\%$
test_step_mdp_speed[True-False-True-False-True] 55.5640μs 15.2138μs 65.7298 KOps/s 64.3426 KOps/s $\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-False-True-False-False] 39.7450μs 9.5446μs 104.7718 KOps/s 99.5583 KOps/s $\textbf{\color{#35bf28}+5.24\%}$
test_step_mdp_speed[True-False-False-True-True] 86.7130μs 26.4480μs 37.8101 KOps/s 37.0769 KOps/s $\color{#35bf28}+1.98\%$
test_step_mdp_speed[True-False-False-True-False] 48.5110μs 17.2359μs 58.0183 KOps/s 57.1005 KOps/s $\color{#35bf28}+1.61\%$
test_step_mdp_speed[True-False-False-False-True] 41.3480μs 16.0476μs 62.3146 KOps/s 61.2224 KOps/s $\color{#35bf28}+1.78\%$
test_step_mdp_speed[True-False-False-False-False] 44.0730μs 10.5574μs 94.7204 KOps/s 93.0646 KOps/s $\color{#35bf28}+1.78\%$
test_step_mdp_speed[False-True-True-True-True] 54.1120μs 25.2365μs 39.6252 KOps/s 38.8743 KOps/s $\color{#35bf28}+1.93\%$
test_step_mdp_speed[False-True-True-True-False] 61.3950μs 15.8600μs 63.0516 KOps/s 61.4092 KOps/s $\color{#35bf28}+2.67\%$
test_step_mdp_speed[False-True-True-False-True] 53.2200μs 17.0784μs 58.5537 KOps/s 57.4952 KOps/s $\color{#35bf28}+1.84\%$
test_step_mdp_speed[False-True-True-False-False] 34.3640μs 10.5691μs 94.6157 KOps/s 93.4061 KOps/s $\color{#35bf28}+1.29\%$
test_step_mdp_speed[False-True-False-True-True] 64.3810μs 26.0916μs 38.3265 KOps/s 37.4620 KOps/s $\color{#35bf28}+2.31\%$
test_step_mdp_speed[False-True-False-True-False] 0.1141ms 17.7674μs 56.2830 KOps/s 56.6058 KOps/s $\color{#d91a1a}-0.57\%$
test_step_mdp_speed[False-True-False-False-True] 46.3670μs 18.2345μs 54.8410 KOps/s 53.8008 KOps/s $\color{#35bf28}+1.93\%$
test_step_mdp_speed[False-True-False-False-False] 36.2380μs 11.7749μs 84.9262 KOps/s 83.4207 KOps/s $\color{#35bf28}+1.80\%$
test_step_mdp_speed[False-False-True-True-True] 67.3670μs 27.5541μs 36.2923 KOps/s 35.8412 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[False-False-True-True-False] 53.4300μs 18.5630μs 53.8706 KOps/s 52.2579 KOps/s $\color{#35bf28}+3.09\%$
test_step_mdp_speed[False-False-True-False-True] 46.0260μs 18.2639μs 54.7529 KOps/s 53.7326 KOps/s $\color{#35bf28}+1.90\%$
test_step_mdp_speed[False-False-True-False-False] 77.9760μs 12.0160μs 83.2224 KOps/s 81.3431 KOps/s $\color{#35bf28}+2.31\%$
test_step_mdp_speed[False-False-False-True-True] 62.5280μs 28.5652μs 35.0076 KOps/s 34.6084 KOps/s $\color{#35bf28}+1.15\%$
test_step_mdp_speed[False-False-False-True-False] 3.5814ms 19.6839μs 50.8028 KOps/s 50.2767 KOps/s $\color{#35bf28}+1.05\%$
test_step_mdp_speed[False-False-False-False-True] 56.9070μs 19.3138μs 51.7763 KOps/s 51.2777 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[False-False-False-False-False] 91.9220μs 12.8918μs 77.5685 KOps/s 76.8348 KOps/s $\color{#35bf28}+0.95\%$
test_values[generalized_advantage_estimate-True-True] 12.5958ms 12.1275ms 82.4570 Ops/s 82.8755 Ops/s $\color{#d91a1a}-0.50\%$
test_values[vec_generalized_advantage_estimate-True-True] 29.6932ms 27.0544ms 36.9626 Ops/s 37.1271 Ops/s $\color{#d91a1a}-0.44\%$
test_values[td0_return_estimate-False-False] 0.2503ms 0.1846ms 5.4176 KOps/s 5.2306 KOps/s $\color{#35bf28}+3.57\%$
test_values[td1_return_estimate-False-False] 27.4010ms 26.1068ms 38.3041 Ops/s 38.7151 Ops/s $\color{#d91a1a}-1.06\%$
test_values[vec_td1_return_estimate-False-False] 59.1113ms 29.1509ms 34.3043 Ops/s 37.0920 Ops/s $\textbf{\color{#d91a1a}-7.52\%}$
test_values[td_lambda_return_estimate-True-False] 39.5949ms 36.5558ms 27.3554 Ops/s 27.3570 Ops/s $-0.01\%$
test_values[vec_td_lambda_return_estimate-True-False] 36.7762ms 27.3424ms 36.5732 Ops/s 36.9070 Ops/s $\color{#d91a1a}-0.90\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.5144ms 8.2838ms 120.7169 Ops/s 125.5552 Ops/s $\color{#d91a1a}-3.85\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 10.1785ms 2.0282ms 493.0470 Ops/s 540.3755 Ops/s $\textbf{\color{#d91a1a}-8.76\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.5712ms 0.4335ms 2.3067 KOps/s 2.2438 KOps/s $\color{#35bf28}+2.80\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 48.8926ms 41.5851ms 24.0471 Ops/s 24.0453 Ops/s $+0.01\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 11.1551ms 2.5934ms 385.6003 Ops/s 391.0177 Ops/s $\color{#d91a1a}-1.39\%$
test_dqn_speed 9.6636ms 1.6306ms 613.2897 Ops/s 598.0734 Ops/s $\color{#35bf28}+2.54\%$
test_ddpg_speed 70.5358ms 3.8886ms 257.1637 Ops/s 268.4252 Ops/s $\color{#d91a1a}-4.20\%$
test_sac_speed 19.6605ms 10.3811ms 96.3293 Ops/s 95.2308 Ops/s $\color{#35bf28}+1.15\%$
test_redq_speed 28.0531ms 19.7165ms 50.7190 Ops/s 46.0802 Ops/s $\textbf{\color{#35bf28}+10.07\%}$
test_redq_deprec_speed 24.3397ms 15.6511ms 63.8933 Ops/s 63.4842 Ops/s $\color{#35bf28}+0.64\%$
test_td3_speed 18.3716ms 10.6273ms 94.0970 Ops/s 92.7044 Ops/s $\color{#35bf28}+1.50\%$
test_cql_speed 46.8275ms 38.8169ms 25.7620 Ops/s 25.6659 Ops/s $\color{#35bf28}+0.37\%$
test_a2c_speed 16.3290ms 8.3596ms 119.6234 Ops/s 122.1672 Ops/s $\color{#d91a1a}-2.08\%$
test_ppo_speed 17.6756ms 8.6614ms 115.4554 Ops/s 117.8676 Ops/s $\color{#d91a1a}-2.05\%$
test_reinforce_speed 16.2194ms 7.4543ms 134.1499 Ops/s 136.7534 Ops/s $\color{#d91a1a}-1.90\%$
test_iql_speed 43.8759ms 34.6873ms 28.8290 Ops/s 28.6919 Ops/s $\color{#35bf28}+0.48\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 2.3748ms 1.9093ms 523.7546 Ops/s 521.8334 Ops/s $\color{#35bf28}+0.37\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.2206ms 2.0948ms 477.3676 Ops/s 489.9670 Ops/s $\color{#d91a1a}-2.57\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.4099ms 2.0925ms 477.8936 Ops/s 481.8153 Ops/s $\color{#d91a1a}-0.81\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.8770ms 1.9289ms 518.4215 Ops/s 519.1230 Ops/s $\color{#d91a1a}-0.14\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.5604ms 2.0896ms 478.5696 Ops/s 488.7607 Ops/s $\color{#d91a1a}-2.09\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.4691ms 2.1041ms 475.2620 Ops/s 486.4268 Ops/s $\color{#d91a1a}-2.30\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.0791ms 1.9008ms 526.1034 Ops/s 511.2014 Ops/s $\color{#35bf28}+2.92\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.2073ms 2.0655ms 484.1512 Ops/s 494.3654 Ops/s $\color{#d91a1a}-2.07\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.0001ms 2.0953ms 477.2640 Ops/s 479.2652 Ops/s $\color{#d91a1a}-0.42\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.9333ms 2.0725ms 482.5188 Ops/s 511.9925 Ops/s $\textbf{\color{#d91a1a}-5.76\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.2948ms 2.1232ms 470.9800 Ops/s 428.7532 Ops/s $\textbf{\color{#35bf28}+9.85\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.8066ms 2.0766ms 481.5624 Ops/s 484.2339 Ops/s $\color{#d91a1a}-0.55\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.5995ms 1.9028ms 525.5400 Ops/s 516.0127 Ops/s $\color{#35bf28}+1.85\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.0803ms 2.1048ms 475.1009 Ops/s 442.4543 Ops/s $\textbf{\color{#35bf28}+7.38\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.6382ms 2.0841ms 479.8162 Ops/s 479.6149 Ops/s $\color{#35bf28}+0.04\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.4320ms 1.9658ms 508.6939 Ops/s 515.9570 Ops/s $\color{#d91a1a}-1.41\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.1186ms 2.0377ms 490.7511 Ops/s 438.2522 Ops/s $\textbf{\color{#35bf28}+11.98\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.3723ms 2.1190ms 471.9191 Ops/s 476.7122 Ops/s $\color{#d91a1a}-1.01\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.1648s 17.6102ms 56.7852 Ops/s 55.3291 Ops/s $\color{#35bf28}+2.63\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1068s 16.3464ms 61.1755 Ops/s 60.2557 Ops/s $\color{#35bf28}+1.53\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1176s 16.6174ms 60.1778 Ops/s 60.6476 Ops/s $\color{#d91a1a}-0.77\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1155s 16.8098ms 59.4891 Ops/s 60.2389 Ops/s $\color{#d91a1a}-1.24\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1174s 16.9969ms 58.8344 Ops/s 60.8407 Ops/s $\color{#d91a1a}-3.30\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1148s 16.9554ms 58.9784 Ops/s 60.1354 Ops/s $\color{#d91a1a}-1.92\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1209s 17.2115ms 58.1008 Ops/s 60.1386 Ops/s $\color{#d91a1a}-3.39\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1149s 16.9357ms 59.0468 Ops/s 59.5426 Ops/s $\color{#d91a1a}-0.83\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1174s 16.8949ms 59.1896 Ops/s 59.8852 Ops/s $\color{#d91a1a}-1.16\%$

@github-actions GitHub Actions
Copy link

github-actions bot commented Nov 29, 2023 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_single 0.1229s 0.1216s 8.2261 Ops/s 8.2255 Ops/s $+0.01\%$
test_sync 0.1033s 0.1028s 9.7268 Ops/s 9.7094 Ops/s $\color{#35bf28}+0.18\%$
test_async 0.2804s 0.1009s 9.9148 Ops/s 10.0190 Ops/s $\color{#d91a1a}-1.04\%$
test_single_pixels 0.1452s 0.1445s 6.9195 Ops/s 6.9528 Ops/s $\color{#d91a1a}-0.48\%$
test_sync_pixels 96.9551ms 95.6214ms 10.4579 Ops/s 10.5817 Ops/s $\color{#d91a1a}-1.17\%$
test_async_pixels 0.2439s 91.0323ms 10.9851 Ops/s 10.9647 Ops/s $\color{#35bf28}+0.19\%$
test_simple 0.9726s 0.9168s 1.0907 Ops/s 1.1390 Ops/s $\color{#d91a1a}-4.24\%$
test_transformed 1.2370s 1.1712s 0.8538 Ops/s 0.8880 Ops/s $\color{#d91a1a}-3.85\%$
test_serial 2.6121s 2.5108s 0.3983 Ops/s 0.4085 Ops/s $\color{#d91a1a}-2.50\%$
test_parallel 2.5613s 2.4985s 0.4002 Ops/s 0.4008 Ops/s $\color{#d91a1a}-0.15\%$
test_step_mdp_speed[True-True-True-True-True] 73.0510μs 35.1923μs 28.4153 KOps/s 27.7517 KOps/s $\color{#35bf28}+2.39\%$
test_step_mdp_speed[True-True-True-True-False] 46.7910μs 20.6009μs 48.5416 KOps/s 47.5548 KOps/s $\color{#35bf28}+2.08\%$
test_step_mdp_speed[True-True-True-False-True] 41.7510μs 21.0448μs 47.5176 KOps/s 47.7728 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-True-True-False-False] 28.5700μs 12.1157μs 82.5375 KOps/s 80.6843 KOps/s $\color{#35bf28}+2.30\%$
test_step_mdp_speed[True-True-False-True-True] 79.7310μs 37.1144μs 26.9437 KOps/s 26.5774 KOps/s $\color{#35bf28}+1.38\%$
test_step_mdp_speed[True-True-False-True-False] 48.4510μs 22.2547μs 44.9343 KOps/s 43.8160 KOps/s $\color{#35bf28}+2.55\%$
test_step_mdp_speed[True-True-False-False-True] 42.5510μs 22.4583μs 44.5271 KOps/s 44.2858 KOps/s $\color{#35bf28}+0.54\%$
test_step_mdp_speed[True-True-False-False-False] 33.4610μs 13.9100μs 71.8906 KOps/s 70.7420 KOps/s $\color{#35bf28}+1.62\%$
test_step_mdp_speed[True-False-True-True-True] 66.5110μs 39.6265μs 25.2356 KOps/s 25.1582 KOps/s $\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-False-True-True-False] 47.9210μs 24.6450μs 40.5762 KOps/s 40.5101 KOps/s $\color{#35bf28}+0.16\%$
test_step_mdp_speed[True-False-True-False-True] 40.3510μs 22.6878μs 44.0766 KOps/s 43.3957 KOps/s $\color{#35bf28}+1.57\%$
test_step_mdp_speed[True-False-True-False-False] 30.3200μs 13.9199μs 71.8395 KOps/s 70.2144 KOps/s $\color{#35bf28}+2.31\%$
test_step_mdp_speed[True-False-False-True-True] 66.3810μs 40.5062μs 24.6876 KOps/s 24.5046 KOps/s $\color{#35bf28}+0.75\%$
test_step_mdp_speed[True-False-False-True-False] 42.1300μs 26.0529μs 38.3834 KOps/s 37.5996 KOps/s $\color{#35bf28}+2.08\%$
test_step_mdp_speed[True-False-False-False-True] 42.2210μs 24.6554μs 40.5590 KOps/s 41.1845 KOps/s $\color{#d91a1a}-1.52\%$
test_step_mdp_speed[True-False-False-False-False] 33.6110μs 15.9010μs 62.8890 KOps/s 62.5256 KOps/s $\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-True-True-True-True] 70.0110μs 38.9052μs 25.7035 KOps/s 25.1498 KOps/s $\color{#35bf28}+2.20\%$
test_step_mdp_speed[False-True-True-True-False] 49.8700μs 24.4048μs 40.9755 KOps/s 39.6258 KOps/s $\color{#35bf28}+3.41\%$
test_step_mdp_speed[False-True-True-False-True] 53.7410μs 26.6862μs 37.4726 KOps/s 37.1666 KOps/s $\color{#35bf28}+0.82\%$
test_step_mdp_speed[False-True-True-False-False] 50.2100μs 15.9876μs 62.5485 KOps/s 61.5377 KOps/s $\color{#35bf28}+1.64\%$
test_step_mdp_speed[False-True-False-True-True] 78.0620μs 42.0881μs 23.7597 KOps/s 24.1305 KOps/s $\color{#d91a1a}-1.54\%$
test_step_mdp_speed[False-True-False-True-False] 67.6910μs 26.5237μs 37.7022 KOps/s 37.0614 KOps/s $\color{#35bf28}+1.73\%$
test_step_mdp_speed[False-True-False-False-True] 47.7100μs 29.0031μs 34.4790 KOps/s 34.6041 KOps/s $\color{#d91a1a}-0.36\%$
test_step_mdp_speed[False-True-False-False-False] 75.8010μs 17.7349μs 56.3859 KOps/s 55.4869 KOps/s $\color{#35bf28}+1.62\%$
test_step_mdp_speed[False-False-True-True-True] 62.9510μs 42.7804μs 23.3752 KOps/s 23.0472 KOps/s $\color{#35bf28}+1.42\%$
test_step_mdp_speed[False-False-True-True-False] 45.3700μs 28.0964μs 35.5917 KOps/s 34.9351 KOps/s $\color{#35bf28}+1.88\%$
test_step_mdp_speed[False-False-True-False-True] 71.6410μs 28.8953μs 34.6077 KOps/s 35.0415 KOps/s $\color{#d91a1a}-1.24\%$
test_step_mdp_speed[False-False-True-False-False] 42.1100μs 17.4232μs 57.3949 KOps/s 55.8001 KOps/s $\color{#35bf28}+2.86\%$
test_step_mdp_speed[False-False-False-True-True] 85.2120μs 44.0494μs 22.7018 KOps/s 22.7311 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[False-False-False-True-False] 56.8410μs 30.0912μs 33.2323 KOps/s 32.5202 KOps/s $\color{#35bf28}+2.19\%$
test_step_mdp_speed[False-False-False-False-True] 72.1010μs 30.1872μs 33.1266 KOps/s 32.8009 KOps/s $\color{#35bf28}+0.99\%$
test_step_mdp_speed[False-False-False-False-False] 41.0210μs 19.6134μs 50.9855 KOps/s 50.4653 KOps/s $\color{#35bf28}+1.03\%$
test_values[generalized_advantage_estimate-True-True] 25.5640ms 25.1359ms 39.7838 Ops/s 40.1876 Ops/s $\color{#d91a1a}-1.00\%$
test_values[vec_generalized_advantage_estimate-True-True] 86.7363ms 3.2932ms 303.6552 Ops/s 304.9340 Ops/s $\color{#d91a1a}-0.42\%$
test_values[td0_return_estimate-False-False] 0.1028ms 67.5450μs 14.8049 KOps/s 15.1913 KOps/s $\color{#d91a1a}-2.54\%$
test_values[td1_return_estimate-False-False] 57.3057ms 55.7678ms 17.9315 Ops/s 18.4608 Ops/s $\color{#d91a1a}-2.87\%$
test_values[vec_td1_return_estimate-False-False] 2.0377ms 1.7458ms 572.8072 Ops/s 580.5355 Ops/s $\color{#d91a1a}-1.33\%$
test_values[td_lambda_return_estimate-True-False] 90.7970ms 89.3277ms 11.1947 Ops/s 10.9251 Ops/s $\color{#35bf28}+2.47\%$
test_values[vec_td_lambda_return_estimate-True-False] 2.0122ms 1.7337ms 576.7972 Ops/s 581.9462 Ops/s $\color{#d91a1a}-0.88\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.5635ms 23.9340ms 41.7815 Ops/s 40.5597 Ops/s $\color{#35bf28}+3.01\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 0.8816ms 0.7215ms 1.3860 KOps/s 1.3931 KOps/s $\color{#d91a1a}-0.51\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7499ms 0.6801ms 1.4704 KOps/s 1.4821 KOps/s $\color{#d91a1a}-0.79\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5480ms 1.4759ms 677.5608 Ops/s 679.9627 Ops/s $\color{#d91a1a}-0.35\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.9578ms 0.7083ms 1.4117 KOps/s 1.4211 KOps/s $\color{#d91a1a}-0.66\%$
test_dqn_speed 7.9640ms 1.4676ms 681.3864 Ops/s 684.9125 Ops/s $\color{#d91a1a}-0.51\%$
test_ddpg_speed 4.7597ms 3.3303ms 300.2736 Ops/s 298.5705 Ops/s $\color{#35bf28}+0.57\%$
test_sac_speed 10.0790ms 9.2935ms 107.6023 Ops/s 108.2735 Ops/s $\color{#d91a1a}-0.62\%$
test_redq_speed 17.5870ms 16.5784ms 60.3194 Ops/s 59.3600 Ops/s $\color{#35bf28}+1.62\%$
test_redq_deprec_speed 14.4330ms 13.1036ms 76.3147 Ops/s 76.3024 Ops/s $\color{#35bf28}+0.02\%$
test_td3_speed 18.4127ms 9.5057ms 105.1996 Ops/s 105.6919 Ops/s $\color{#d91a1a}-0.47\%$
test_cql_speed 34.1241ms 32.5671ms 30.7059 Ops/s 31.1298 Ops/s $\color{#d91a1a}-1.36\%$
test_a2c_speed 0.1071s 8.0567ms 124.1205 Ops/s 136.3454 Ops/s $\textbf{\color{#d91a1a}-8.97\%}$
test_ppo_speed 8.8128ms 7.5893ms 131.7653 Ops/s 130.8808 Ops/s $\color{#35bf28}+0.68\%$
test_reinforce_speed 7.7075ms 6.2932ms 158.9020 Ops/s 157.2707 Ops/s $\color{#35bf28}+1.04\%$
test_iql_speed 28.9283ms 27.3499ms 36.5632 Ops/s 36.2667 Ops/s $\color{#35bf28}+0.82\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 3.2172ms 2.4876ms 401.9867 Ops/s 396.2246 Ops/s $\color{#35bf28}+1.45\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.9329ms 2.6990ms 370.5080 Ops/s 334.3932 Ops/s $\textbf{\color{#35bf28}+10.80\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.8494ms 2.6937ms 371.2401 Ops/s 371.1459 Ops/s $\color{#35bf28}+0.03\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 2.7054ms 2.4762ms 403.8496 Ops/s 399.8230 Ops/s $\color{#35bf28}+1.01\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 4.3842ms 2.6841ms 372.5587 Ops/s 371.2138 Ops/s $\color{#35bf28}+0.36\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 3.6805ms 2.6823ms 372.8187 Ops/s 372.2062 Ops/s $\color{#35bf28}+0.16\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 2.7445ms 2.5063ms 398.9937 Ops/s 396.4230 Ops/s $\color{#35bf28}+0.65\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 4.0297ms 2.6781ms 373.3933 Ops/s 370.6536 Ops/s $\color{#35bf28}+0.74\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 3.7911ms 2.6942ms 371.1641 Ops/s 372.3999 Ops/s $\color{#d91a1a}-0.33\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 0.1921s 3.0060ms 332.6664 Ops/s 395.4346 Ops/s $\textbf{\color{#d91a1a}-15.87\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 3.5578ms 2.6946ms 371.1058 Ops/s 371.6257 Ops/s $\color{#d91a1a}-0.14\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 3.9798ms 2.7006ms 370.2944 Ops/s 373.9180 Ops/s $\color{#d91a1a}-0.97\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 0.1917s 2.9931ms 334.1045 Ops/s 395.5867 Ops/s $\textbf{\color{#d91a1a}-15.54\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.5369ms 2.6830ms 372.7184 Ops/s 371.3249 Ops/s $\color{#35bf28}+0.38\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 4.1961ms 2.6917ms 371.5112 Ops/s 372.9928 Ops/s $\color{#d91a1a}-0.40\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 0.1913s 3.0077ms 332.4755 Ops/s 396.4501 Ops/s $\textbf{\color{#d91a1a}-16.14\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 4.0278ms 2.6931ms 371.3173 Ops/s 371.6405 Ops/s $\color{#d91a1a}-0.09\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 4.0700ms 2.6782ms 373.3855 Ops/s 371.5540 Ops/s $\color{#35bf28}+0.49\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.2323s 17.4910ms 57.1723 Ops/s 51.5188 Ops/s $\textbf{\color{#35bf28}+10.97\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 0.1268s 17.6035ms 56.8070 Ops/s 56.2724 Ops/s $\color{#35bf28}+0.95\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 0.1280s 17.5385ms 57.0173 Ops/s 56.5135 Ops/s $\color{#35bf28}+0.89\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.1279s 17.5181ms 57.0840 Ops/s 64.6269 Ops/s $\textbf{\color{#d91a1a}-11.67\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 0.1260s 17.5854ms 56.8655 Ops/s 56.3542 Ops/s $\color{#35bf28}+0.91\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 0.1264s 15.2863ms 65.4180 Ops/s 56.5237 Ops/s $\textbf{\color{#35bf28}+15.74\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.1282s 17.5775ms 56.8909 Ops/s 56.1992 Ops/s $\color{#35bf28}+1.23\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 0.1260s 17.4850ms 57.1918 Ops/s 64.8546 Ops/s $\textbf{\color{#d91a1a}-11.82\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 0.1265s 17.6209ms 56.7508 Ops/s 56.5349 Ops/s $\color{#35bf28}+0.38\%$

@vmoens vmoens added the enhancement New feature or request label Nov 29, 2023
@vmoens vmoens added the Data Data-related PR, will launch data-related jobs label Nov 29, 2023
@vmoens vmoens marked this pull request as ready for review November 29, 2023 18:05
@vmoens vmoens merged commit a0fc469 into main Dec 4, 2023
45 of 62 checks passed
@vmoens vmoens deleted the minari branch December 4, 2023 13:56
@vmoens vmoens mentioned this pull request Dec 20, 2023
Open
36 tasks
@alexdavey alexdavey mentioned this pull request Jan 12, 2024
Merged
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Data Data-related PR, will launch data-related jobs enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
2 participants
@vmoens @facebook-github-bot